Agent apparatus

ABSTRACT

On a vehicle, a device for estimating a present position such as a GPS device and a communication device such as a cell phone are installed. The vehicle transmits a detected present position to an information center at specified intervals. On the other hand, the information center sets a plurality of areas in a region to which information is transmitted, and sets information for transmission for each area. When receiving data indicating the present position of the vehicle from the vehicle, the information center specifies, by using the data, an area where the vehicle is located, and transmits the information set for the area to the vehicle.

TECHNICAL FIELD

[0001] This invention relates to an agent apparatus and, specifically,to an agent apparatus which automatically collects information on avehicle driver through conversation with an agent.

BACKGROUND ART

[0002] An agent apparatus to present an agent within a vehicle whichactively conducts communication such as conversation with a user bysensing a state of the vehicle is proposed in the Japanese PatentLaid-open No. Hei 11-37766.

[0003] In the agent apparatus, for example, the remaining quantity offuel such as gasoline, light oil, and the like is detected, suchdetected values for the past five times immediately prior to refuelingare stored, and when the average value thereof is reached, a time torefuel is notified by the agent.

[0004] In such an agent apparatus, the more information on a user iscollected, the more information desired by the user can be provided orthe more conversations can be carried out. Accordingly, it is importantfor an agent apparatus to collect as much information on a driver aspossible.

[0005] However, in various agent apparatuses previously proposed, theuser himself/herself inputs each item on hobbies or preferences asdriver information at an initial state in which the use of the agentapparatus is commenced.

[0006] As described above, since the prior apparatuses are dependent onthe inputting operation by the user himself/herself to collect thedriver information, the load by the inputting operation by the user islarge, and there is even a case in which only the bare minimum ofinformation (essential input items without which a startup isimpossible) is obtained.

[0007] The present invention is made in order to solve the aboveproblem, and it is an object of the present invention to collectinformation on a driver as much as possible while placing the smallestpossible burden on the driver.

DISCLOSURE OF THE INVENTION

[0008] The present invention attains the aforesaid object by includingin an agent apparatus: a load-on-driver detector to detect a load on adriver; a question outputter to output questions about the driver when alow-load state is detected by the load-on-driver detector; and aninformation obtainer to obtain information on the driver based on thedriver's reply to the question made by the question outputter.

[0009] Further in the present invention, in the agent apparatusdescribed in claim 1, a driver information storage to store driverinformation and a question information storage to store questions toobtain the driver information are provided, and the question outputterreferred to above reads out, from the question information storage, aquestion which corresponds to the driver information not stored in thedriver information storage and outputs the question.

[0010] Furthermore in the present invention, in the agent apparatusdescribed in claim 1 or claim 2, as a state of a driver detected by thecondition detector referred to above, at least one of a state of drivingoperation and a state of equipment manipulation is detected; and a stateof a load on a driver determined by the load-on-driver determinerreferred to above is at least one of a load imposed on the drivingoperation and an extent of vexation felt by the driver.

[0011] Moreover in the present invention, in the agent apparatusdescribed in claim 1, claim 2, or claim 3, an agent presenter to presentan artificial pseudo-living organism within the vehicle is provided, andthe question outputter referred to above poses questions through atleast one of a display screen and voice output.

BRIEF DESCRIPTION OF DRAWINGS

[0012]FIG. 1 is a block diagram showing a structure of an agentapparatus which realizes a driver information collecting apparatus in anembodiment of the present invention.

[0013]FIG. 2. is a block diagram showing an agent processing unit in theagent apparatus.

[0014]FIG. 3 is an explanatory view conceptually showing a dataconfiguration and data contents in a scenario file in the agentapparatus.

[0015]FIG. 4 is an explanatory view conceptually showing a compositionof driver information in the agent apparatus.

[0016]FIG. 5 is a flowchart showing operations of a scenario automaticstartup determination process.

[0017]FIG. 6 is a flowchart showing operations of a scenario drivingprocess.

[0018]FIG. 7 is an explanatory view showing a transition of scenescreens by presenting each scene in a question scenario on hobbies andpreferences (on food) among question scenarios in the agent apparatus.

BEST MODE FOR CARRYING OUT THE INVENTION

[0019] Hereinafter, a preferable embodiment of a driver informationcollecting apparatus of the present invention will be explained indetail with reference to FIGS. 1 to 7.

[0020] (1) Overview of the Embodiment

[0021] In an agent apparatus realizing the driver information collectingapparatus in the present embodiment, a personified agent is presentedthrough a screen image (a two-dimensional image, three-dimensional imagesuch as one via holography, and the like) to let the agent performvarious communications with a driver, or representively conductoperations.

[0022] Driver information pertaining to the driver such as age, gender,and hobbies necessary to perform communications and the like with theagent according to preferences of the driver (user) is obtained andadded by questions by the agent.

[0023] The questions for obtaining the driver information are posed bythe agent by judging the state of the load of driving operation ormanipulation of various equipment by the driver, and when in a low-loadstate, questions related to driver information which has not yet beenobtained are outputted (through voice and image) in the order of highestpriority.

[0024] By the driver's replies to the questions posed by the agent, thedriver information is obtained and stored, which is then reflected incommunications with the driver and the behavior of the agent.

[0025] For example, when the driver travels a straight road at aconstant speed, not by particularly listening to music or the like for apredetermined period of time or more, the agent asks the driver aquestion of what type of cuisine he/she likes (driver information of thehighest priority which has not been obtained). The replying result fromthe driver is stored in the driver information (data on hobbies andpreferences), and thereafter the stored data on hobbies and preferencesare reflected when recommending a restaurant (meal) to the driver insuch a manner that the priority for recommending a restaurant of thegenre preferred by the driver becomes high.

[0026] As described above, by outputting questions pertaining to driverinformation as a part of communications with the agent, the driverinformation can be obtained automatically, and thereby the driver isreleased from the vexation of inputting all of the driver information inadvance.

[0027] Further, since the questions pertaining to the driver informationare outputted by selecting a timing at which the load on the driver isless, a question is not outputted when the driver is manipulatingequipment such as an air conditioner, performing a driving operationincluding steering, listening to an audio equipment or the radio, orchatting with someone in the vehicle, so that the vexation felt by thedriver can be avoided.

[0028] In addition, since the driver obtains and stores the driverinformation automatically while communicating with the agent, the morethe driver answers to questions, the more information and communicationsuitable for the driver's hobbies and preferences the agent apparatuscan provide.

[0029] Additionally, the agent in the present embodiment is anartificial pseudo-living organism, and its figure (two-dimensionalimage, three-dimensional image like holography, and so forth) ispresented within the vehicle by an image display apparatus. Theproceedings by the agent are determining on and learning of the state ofthe vehicle regarding the vehicle itself, passengers, oncoming vehicles,and so forth (the learning not only on the state but also on answers andreplies from the driver), in which, based on the state of the vehicle ateach time point and the learning outcome up to that time point, theagent responds with wide variations (its behavior=action and voice) tothe driver and the vehicle. This allows the driver to call up aplurality of agents within the vehicle at liberty and make thein-vehicle environment comfortable.

[0030] Here, the artificial pseudo-living organism (agent) in thepresent embodiment has identity of subject such as a specific person, acreature, a comic character and the like, and such a pseudo-livingorganism having the identity performs outputting (responds with actionand voice) in such a manner that the identity and continuity of thesubject is maintained. Further, the agent in the present embodimentwhose identity and continuity are expressed as something with a uniquecharacter, and which is presented in the vehicle, has a produced voice,image and the like that vary according to the contents learned in thepast even in the same vehicle state.

[0031] (2) Details on the Embodiment

[0032]FIG. 1 is a block diagram showing a structure of the agentapparatus which realizes the driver information collecting apparatus inthe present embodiment.

[0033] In the present embodiment, an entire processing unit 1controlling the entire communication functions is provided. The entireprocessing unit includes: a navigation processing unit 10, to search aroute to the set destination and perform guidance through voice ordisplayed image; an agent processing unit 11; an external I/F unit 12,to correspond to the navigation processing unit 10 and the agentprocessing unit 11; an image processing unit 13, in which outputting ofimages such as an agent image, map image, and the like, as well asinputted images are processed; a voice control unit 14, in whichoutputting of voice such as the voice of the agent and the routeguidance voice as well as inputted voice are controlled; a stateinformation processing unit 15, to process the detected data on variousstates on the vehicle or the driver; an input control unit 16; and astorage apparatus control unit 17.

[0034] The agent processing unit 11 determines the load on the driverfrom the state inside and outside of the vehicle, the operating state bythe driver, and so forth, and asks questions to obtain the driverinformation which has not been accumulated when the load on the driveris low, to obtain and save the driver information from the reply asdescribed later.

[0035] The navigation processing unit 10 and the agent processing unit11 include a CPU (central processing unit) to control the dataprocessing and the operations of each unit, and a ROM, a RAM, a timerand so forth connected to the CPU through bus lines such as a data bus,a control bus, and the like. The both processing units 10 and 11 areconnected by a network (in-vehicle LAN (local area network)) in a mannerthat the processed data can be obtained from each other.

[0036] The ROM is a read-only memory to store in advance various dataand programs for performing control in the CPU, and the RAM is a randomaccess memory which the CPU uses as a working memory.

[0037] In the navigation processing unit 10 and the agent processingunit 11 in the present embodiment, the CPU reads the various programsstored in the ROM, and carries out various processings. Note that theCPU may read a computer program from an external storage medium set inthe storage medium drive apparatus 23, store (install) the program inanother storage apparatus such as an agent data storage apparatus 30,navigation data storage apparatus 31 or a hard disk not shown, read(load) a necessary program or the like from the storage media into theRAM, and execute it. Further, a necessary program or the like may beexecuted by being read directly from the storage medium drive apparatus23 into the RAM.

[0038] In the present embodiment, the agent processing unit 11 createsvarious communicating behaviors of the agent including conversation withthe driver and the handling operations to be performed in the navigationprocessing section 10, by assuming various states (cases) on the vehicleor the driver as well as driver information outcomes, and performs themaccording to a scenario. In other words, where various states such asvehicle speed, time, traveling region, temperature, remaining volume ofgasoline, psychological distance, and driver information are thescenario startup conditions and the scene branching conditions, theactions to be taken by the agent for each condition are defined as ascenario for each condition.

[0039] Each scenario is consisted of a plurality of continuous scenes. Ascene is a situated stage in a scenario, in which the agent poses acertain question, or provides information. Each scene is composed of atitle, list, word balloon, background, and other small units (parts).

[0040] Each scene is preceded sequentially according to the scenario.Depending on the scenario, a plurality of selective scenes existaccording to the driver's reply to a question posed in a certain scene,the situation of the vehicle, and the like. In other words, there arescenarios in which a scene branches according to a reply in the courseof the scenario.

[0041] The data on the scenarios including the scenes are stored in thescenario data file 302 which will be described later. Summarized by eachscene and stored as scenario data 1 to n of the scenario data file 302are: the information to define when and where to execute a scenario(scenario startup condition), and when the scenario is executed, thedata to define what behavior or conversation is performed by the agentand how the screen is arranged, what sort of instruction is issued to amodule such as the navigation processing unit 10 and so forth, and whatwill be performed in the following (which scene should be selected) whenreceiving an event.

[0042]FIG. 2 shows a structure of the agent processing unit 11.

[0043] As shown in FIG. 2, the agent processing unit 11 includes a statedetermining unit 11, a scenario execution request determining unit 112,a scenario drive unit 113, a scenario data read-in unit 114, an imageprocessing unit 115, a voice recognition control unit 116, a user inputreception unit 117, and a data record unit 118.

[0044] Each of the units 111 to 118 are realized by a correspondingengine (software) stored in the CPU, the ROM, and so forth provided inthe agent processing unit 11. The engine constituting each unit such asthe state determination unit 111 and the like issue a correspondingevent when transmitting data to other engine. The events issued by units111 to 118 does not include an event to determine whether a scene shouldbranch or not.

[0045] The state determining unit 111 is equipped with an API(application programming interface) notifying of a change in a state oftime, place, various inputs, or the like. The state determining unit 111issues an event to the scenario execution request determining unit 112according to a change in the states (the set destination or the like).

[0046] The scenario execution request determining unit 112 refers to,through the API function, the determined outcome of the states, anddetermines whether or not to execute a scenario. When the scenario isexecuted, an event to request a startup of the scenario is issued to thescenario drive unit 113.

[0047] With the scenario startup request issued by the scenarioexecution request determining unit 112, the scenario drive unit 113determines whether or not to perform the startup.

[0048] When it is determined to perform the startup, the scenario driveunit 113 requests the scenario data read-in unit 114 to read thescenario data. Further, according to the scenario data, the scenariodriver unit 113 forwards a request on image processing and a request onthe character's motions to the image processing unit 115. The scenariodriver unit 113 also notifies the voice recognition control unit 116 ofa dictionary used for the voice reorganization according to the scenariodata. Further, the scenario drive unit 113 drives the scenario (controlsthe transition of scenes) by the voice recognition result or inputs bythe user.

[0049] In addition, the scenario drive unit 113 drives the scenario(controls the transition of scenes) by referring to the records such asthe driver information by the data record unit 118 through the APIfunction.

[0050] The scenario data read-in unit 114 reads a pertinent scenariodata from the scenario data file 302 based on the request by an eventfrom the scenario drive unit 113. The scenario data read-in unit 114also reads image data by the request by an event from the scenario driveunit 113.

[0051] The image processing unit 115 renders an image of a correspondingscene or of the character (agent) based on the request by an event fromthe scenario drive unit 113.

[0052] The voice recognition control unit 116 controls the instructionsto a voice recognition unit 142. For example, the voice recognitioncontrol unit 116 specifies a dictionary for use which is notified by thescenario drive unit 113. When a voice recognition result by thespecified dictionary for use comes out in the voice recognition unit142, the voice recognition unit 116 issues an event corresponding to therecognized result to the scenario drive unit 113.

[0053] The user input reception unit 117 issues to the scenario driveunit 113 an event corresponding to the inputted content when receivingan input from the user. Whether a user made an input is determined bythe information from the input control unit 16.

[0054] The data record unit 118 includes the API function to record thedriver information and the like, and the API function to notify thedriver information and the like.

[0055] Incidentally, while the units of the agent processing unit 17have been explained for the case of issuing an event and the case ofusing the API function, the event issuing may be applied to all theunits, or the API function usage may be applied to all the units.

[0056] In FIG. 1, the storage medium drive apparatus 23 and acommunication control unit 24 are connected to the external I/F unit 12;a display apparatus 27 and an imaging apparatus 28 are connected to theimage processing unit 13; a voice output apparatus 25 and a microphone(voice obtainer) 26 are connected to the voice control unit 14; avarious states detecting apparatus 40 is connected to the stateinformation processing unit 15; and an input apparatus 22 is connectedto the input control unit 16.

[0057] The various states detecting apparatus 40 includes a presentposition detecting apparatus 41 and a state detecting unit 42. Thecurrent position detector 41 is for detecting an absolute position (bylongitude and latitude) of the vehicle, and a GPS (global positioningsystem) reception apparatus 411 to measure the position of the vehicleby using an artificial satellite, an azimuth sensor 412, a rudder anglesensor 413, a distance sensor 414, a beacon reception apparatus 415 toreceive positional information from a beacon which is arranged over aroad, and so forth are used.

[0058] While the GPS reception apparatus 411 and the beacon receptionapparatus 415 are capable of measuring a location on its own, in a placewhere reception by the GPS reception apparatus 411 or the beaconreception apparatus 415 are impossible, a current position is supposedto be detected by a dead-reckoning navigation method using both theazimuth sensor 412 and the distance sensor 414.

[0059] For the azimuth sensor 412, for example, a geomagnetic sensor toevaluate the azimuth of the vehicle by detecting the geomagnetism, agyro such as a gas-rate gyro and an optic-fiber gyro to evaluate theazimuth of the vehicle by detecting the roll angular velocity of thevehicle and integrating the angular velocity, wheel sensors to calculatethe displacement amount by arranging sensors to the right and the leftwheels and detecting the rounding of the vehicle by the output pulsedifference (moved distance difference), and the like are used.

[0060] The rudder angle sensor 413 detects a steering angle □ by usingan optic rotation sensor installed on the rotating part of a steering, arotation resistance volume, and the like.

[0061] The distance sensor 414 employs various methods, for example, amethod which detects and counts the rotation number of the wheel, or amethod which detects the acceleration and integrates it twice.

[0062] The distance sensor 414 and the rudder angle sensor 413 alsofunction as driving operation state detector.

[0063] The state detecting unit 42 includes a brake sensor 421, avehicle speed sensor 422, a direction indicator detector 423, a shiftlever sensor 424, and a handbrake (parking brake) sensor 425 functioningas the driving operation state detector to detect the state of drivingoperations.

[0064] Further, the state detecting unit 42 includes an air conditionerdetector 427, a wiper detector 428, and an audio detector 429functioning as equipment manipulation state detector to detect the stateof manipulation of equipment.

[0065] The brake sensor 421 detects whether or not the foot brake is ina pressed state.

[0066] The vehicle speed sensor 422 detects the speed of the vehicle.

[0067] The direction indicator detector 423 detects whether the driveris manipulating the direction indicator or not, as well as whether thedirection indicator is flashing or not.

[0068] The shift lever sensor 424 detects whether or not the driver ismanipulating the shift lever, as well as the position of the shiftlever.

[0069] The handbrake (parking brake) sensor 425 detects whether or notthe driver is manipulating the side brake, as well as the state of theside brake (on or off).

[0070] The air conditioner detector 427 detects whether the driver ismanipulating the various switches and the like of the air conditioner.

[0071] The wiper detector 428 detects whether the driver is manipulatingthe wiper or not.

[0072] The audio detector 429 detects whether or not the driver ismanipulating the audio equipment such as a radio, CD player, andcassette player, and whether or not an audio output is being made by theaudio equipment.

[0073] The state detecting unit 42 includes, as other equipmentmanipulation state detector, a light detecting sensor to detect theoperation state of the headlight, dome light, and the like, a seatbeltdetecting sensor to detect the fastening and unfastening of the seatbeltby the driver, and other sensors.

[0074] The input apparatus 22 is for the driver to input the driverinformation, as well as one of the means to respond to all of theremaining questions and the like by the agent in the present embodiment.

[0075] The input apparatus 22 is also for inputting: the currentposition at the start of driving (starting point) and the destination(arrival point) for the navigation processing; the predeterminedtraveling environment of the vehicle in which one would want to transmita request for information such as traffic congestion information to aninformation providing station (transmission condition); the type (model)of the cell phone used in the vehicle; and so forth.

[0076] As the input apparatus 22, various equipment such as a touchpanel (functioning as a switch), keyboard, mouse, light pen, joystick,remote controller by infrared or the like, and audio recognitionequipment may be used. Further, a remote controller using infrared orthe like and a reception unit to receive various signals transmittedfrom the remote controller may also be provided. In the remotecontroller, arranged besides a joystick to perform manipulation such asshifting the cursor displayed on the screen are various keys such as amenu specifying key (button) and a numeric keypad.

[0077] The input control unit 16 detects data corresponding to theinputted content by the input apparatus 22 and supplies it to the agentprocessing unit 11 and the navigation processing unit 10. The inputcontrol unit 16 functions as the equipment manipulation state detectorby detecting whether the driver is in inputting operation or not.

[0078] The storage medium drive apparatus 23 is a drive apparatus to beused to read a computer program from an external storage medium for thenavigation processing unit 10 and the agent processing unit 11 toperform various proceedings. In the computer programs stored in thestorage medium, various programs, data, and the like are included.

[0079] Here, the storage medium means a storage medium to store computerprograms, and more specifically, it includes the storage medium to storecomputer programs, by magnetic storage media such as a floppy disk, harddisk, and magnetic tape, semiconductor storage media such as a memorychip and IC card, storage media capable of reading information opticallysuch as a CD-ROM, MO, PD (phase change rewritable optical disk), andother various methods.

[0080] Besides reading computer programs from these various storagemedia, if the storage media are writable storage media such as a floppydisk or IC card, the storage medium drive apparatus 23 is able to writein such storage media the data of the RAMs in the navigation processingunit 10, and the agent processing unit 11, or of a storage apparatus 29,or the like.

[0081] For example, by storing data such as the learning contents forthe agent functions (learning item data/reply data) or driverinformation in an IC card, even when the driver drives a differentvehicle, such data stored in the IC card can be read out and used, sothat he/she can communicate with the agent which is in a state of havinglearned from his/her past responding states. Hence, it becomes possibleto present, in the vehicle, not an agent for each specific vehicle butthe agent having the learning contents unique to each driver.

[0082] It is also possible to store a scenario data in the IC card.Hereby an original scenario unique to each user is made possible.

[0083] The communication control unit 24 is capable of being connectedto cell phones constituted by various wireless communication equipment.Besides communication over the phone circuit, the communication controlunit 24 is configured to allow communications with the informationprovision station which provides traffic information data such as dataon traffic congestion or traffic regulation, or with the informationprovision station which provides karaoke data to be used for an onlinekaraoke in vehicle.

[0084] Further, it is also possible to transmit and receive through thecommunication control unit 24 the learning data for the agent functions,the scenario data file, and so forth.

[0085] The voice output apparatus 25 is composed of a plurality ofspeakers arranged in the vehicle, to output voice controlled by thevoice control unit 14 such as the guidance voice when a route guidanceis performed through voice, and the voice and sound for the agent'sconversations for regular communications with the driver as well as forthe agent's questions for obtaining driver information according to thepresent embodiment. The voice output apparatus 25 may also serve as anaudio speaker.

[0086] The microphone 26 functions as a voice inputter to input andoutput voice which is the subject of the voice recognition by the voicecontrol unit 14 such as the inputted voice for a destination and thelike in the navigation processing, and conversations between the agentand the driver (including the replies by the driver) and the like. Forthis microphone 26, a dedicated directional microphone so as to properlycollect the voice of the driver is used.

[0087] Note that the voice output apparatus 25 and the microphone 26together may form a hand-free unit to allow phone communication withoutusing the cell phone.

[0088] The microphone 26 and the voice recognition unit 142 function asa conversation detector to detect whether the driver is talking with apassenger, in which case the microphone 26 and the voice recognitionunit 142 function as a state detector to detect the state of the driver.

[0089] The display apparatus 27 displays a road map for route guidanceprocessed by the navigation processing unit 10, as well as various imageinformation, or displays various behaviors (motion pictures) of theagent made by the agent processing unit 11 as well as various parts(components) constituting a screen layout. Further, the images of theinside and outside of the vehicle taken by the imaging apparatus 28 aredisplayed after being processed in the image processing unit 13.

[0090] For the display apparatus 27, various display apparatuses such asa liquid crystal display, CRT, and the like are used.

[0091] Note that the display apparatus 27 may also be equipped with afunction as that of the aforementioned input apparatus 22 such as atouch panel or the like.

[0092] The imaging apparatus 28 is composed of cameras equipped with CCD(charge-coupled device) to take an image, and besides an in-vehiclecamera to take the image of the driver, cameras are arranged outside ofthe vehicle to take images from the front, rear, right side, and leftside of the vehicle. The images taken by each camera of the imagingapparatus 28 are supplied to the image processing unit 13, in which suchprocesses as image recognition are made and each recognized result(whether with or without a passenger, recognitions on the driver, and soforth) are reflected in the communications by the agent.

[0093] The storage apparatus 29 is connected to the storage apparatuscontrol unit 17 so as to read and write data and programs under thecontrol of the storage control apparatus 17.

[0094] The storage apparatus 29 stores agent data 30 and navigation data31 as the various data (including programs) necessary to realize variousagent functions and navigation functions according to the presentembodiment.

[0095] For the storage apparatus 29, for example, various types ofstorage media such as a floppy disk, hard disk, CD-ROM, optical disk,magnetic tape, IC card, optimal card and the like as well as driveapparatus thereof are used.

[0096] In the case hereof, for example, a plurality of different storagemedia and drive apparatus may be the constituents in such a manner that,learning item data 304, reply data 305, driver information 307, andscenario data file 302 are included in IC cards or floppy disks whichare easy to carry, while the other data are included in a hard disk, andthese storage media may also be used as the drive apparatus.

[0097] The agent data 30 stores various types of data which arenecessary for the processings by the agent, such as an agent program301, the scenario data file 302, voice data 303, the learning item data304, the reply data 305 composed of the voice data 303, image data 306to display images of the figure or behavior of the agent, the driverinformation 307, and so forth.

[0098] The agent program 301 stores an agent processing program torealize the agent function.

[0099] The learning item data 304 and the reply data 305 are data tostore the learning result by the agent based on the driving operationsand replies by the driver.

[0100] Accordingly, for the learning item data 304 and the reply data305, the data for each driver are stored and updated (learned).

[0101] The learning item data 304 stores items which are the subjects oflearning for the agent, such as the total number of times or the numberof times per day that the ignition is turned on, and the data on thevolume of remaining fuel at the time of refueling for the last fivetimes. According to the learning content stored in the learning itemdata 304, for example, the content of greetings by the agent at the timeof appearance of the agent changes based on the number of times that theignition is turned on, or, when the volume of the remaining fuel goesbelow the average of the volume of the remaining fuel for the last fivetimes, the agent suggests refueling.

[0102] The reply data 305 stores, for each predetermined scenario, thepast replies by the user to the behavior of the agent. The reply data305 stores, for each reply item, the time and date of the reply and thecontent of the reply, for the predetermined number of times. Respectivecases of being ignored, rejected, or accepted (tolerated) for thecontent of the response are determined based on the voice recognition orthe inputted result in the input apparatus 22, and stored.

[0103] The scenario data file 302 stores the data of scenarios whichspecify behaviors of the agent for each condition or scene.

[0104]FIG. 3 conceptually shows the data structure and the data contentof the scenario data file 302.

[0105] For the scenario data file 302, a plurality of scenario datadefining each scene are aggregated to form one file.

[0106] The scenario data file 302 includes a file header, scenariomanagement table, automatic startup condition table, list of scenarioswhich can be started up manually, various scenario data 1 to n, andadditional resource data.

[0107] The file header is a header for a scenario file and stores theinformation on the file.

[0108] The scenario management table stores the data to manage thescenario data stored in the scenario file, such as the data on thepriorities in executing each scenario data.

[0109] The automatic startup condition table specifies and stores, foreach scenario data, the condition to automatically start up the scenariodata stored in the scenario file.

[0110] As the automatic startup conditions, various automatic startupconditions are stored, such as the data on a specific point and thedistance therefrom for a specific scenario which is performed when apoint at a certain distance from a specific point A is reached, theturning-on of the ignition which is the condition for performing thescenario in which the agent performs a salutation at the time of thestartup, and the predetermined time period after the startup.

[0111] Further, are stored as automatic startup prohibiting conditionsto prohibit an automatic startup, for example: (A) to give priority tomanual startup, (B) not to automatically startup when other scenario isperformed, (C) not to start up a scenario which was performed within,for example, past 30 minutes, (D) not to start up when the music isheard, (E) not to start up when taking a curve to the right or to theleft (when a steering operation is performed), (F) not to start up whenrapidly decelerating or rapidly accelerating, (G) not to start up duringa conversation with a passenger, (H) not to start up when operatingvarious equipment such as a navigation apparatus, audio apparatus,air-conditioning apparatus, shift lever, handbrake, wiper, or directionindicator, (I) not to start up when the audio apparatus is used (acondition for not bothering the listening to the music or a broadcaststation: not to turn on the audio apparatus), and (J) not to start upduring a driving route guidance by the navigation processing unit 10.Note that the scenario management table establishes priorities so as toallow startup in case of emergency (such as in the scenario to makeemergency contact with the fire station or the like), even when theseautomatic startup prohibition conditions are fulfilled. The automaticstartup prohibition conditions include the conditions which are appliedcommonly to each scenario, and the conditions which are applied for eachspecific scenario.

[0112] In the present embodiment, as the conditions to startup thequestion scenario in which the agent poses questions to collect thedriver information, the following conditions as well as other conditionsare defined as the conditions to determine that the state of the load onthe driver is in a low state:

[0113] (a) The question scenario has not been performed within the pastone hour (a condition for not posing questions too frequently).

[0114] (b) Fifteen minutes or more have passed since the ignition isturned on (not a time immediate after the start of the driving).

[0115] (c) No conversation is heard in the vehicle (by collecting soundthrough the microphone 26).

[0116] (d) The vehicle is driving on a straight road (detected by thenavigation data 31 and the current position), and the speed thereof hasnot changed for a predetermined period of time.

[0117] (e) The vehicle is waiting at a stoplight (determined by thenavigation data 31, the current position, and the vehicle speed).

[0118] For these question-startup conditions, whether they areindispensable conditions or just one of them needs to be fulfilled forthe startup is predetermined. For example, when being limited to theabove conditions (a) to (e), the question-startup conditions arefulfilled and the question scenario is started up when (a) to (c) areentirely satisfied, and (d) or (e) is satisfied.

[0119] Even in this case, the question scenario is prohibited to bestarted up when it falls under the automatic-startup prohibitionconditions starting from (A) described above.

[0120] Additionally, the startup determining routine based on theautomatic startup conditions including the question-startup conditionsis to be processed, for example, by a polling process every one minute,or by receiving an event specified in advance. The events specified inadvance are a setting of a destination, reception of an e-mail, or thelike.

[0121] Among the respective conditions referred to above, the load onthe driving operation is determined by the conditions (E), (F), (H),(J), (d), and (e), while the extent of vexation felt by the driver isdetermined by (C), (D), (G), (I), (J), (a), (b), and (c).

[0122] A list of manually startable scenarios is stored as a list ofdata of manually startable scenarios among the data of scenarios storedin the scenario file.

[0123] As additional resource data, the voice data, image data, and thelike to be added to the scenario data for extending the storageapparatus are stored.

[0124] Each of the scenario data 1 to n is composed of a scenarioheader, voice recognition dictionary DB, image data management table,scene management table, each scene data 1 to m, and actual image data.Each scenario includes a plurality of scenes.

[0125] The scenario header is the header section of the scenario dataand stored therein is information on the scenario.

[0126] The voice recognition dictionary DB is a DB of the voicerecognition dictionary used in a scenario, and stores voice recognitiondictionary data.

[0127] The image data management table stores data to specify images tobe used in the scenario. The image of the agent is stored in the imagedata 306, and the images unique to the scenario are stored in the actualimage data which is at the end of the scenario data.

[0128] The scene data management table stores data to manage the scenedata stored in the scenario.

[0129] Each of the scene data 1 to m is composed of a scene header,screen data, character motion instruction data, various processingsinstruction data, and scene branching table.

[0130] The scene header is the header section of the scene data, and arestored therein data to manage information on the scene and each datasection belonging to the scene data.

[0131] The screen layout data is the section to specify the screenlayout displayed in a certain scene, and are stored therein data on eachpart of the screen layout displayed on the display apparatus 27.

[0132] In the character motion instruction data, the instruction data onthe motion and the content of speech by the character in a pertinentscene (data to instruct the voice data 303 to be used) are stored.

[0133] In the various processings instruction data, data to request aprocessing instruction to other modules (such as a navigator) in apertinent scene are stored.

[0134] The scene branching table is a section which stores branchinginformation from a pertinent scene to the subsequent scene, and storesdata defining the subsequent scene when receiving an event duringdisplay of certain scene.

[0135] In the voice data 303 of the storage apparatus 29 (FIG. 1), voicedata for the agent to conduct conversations and the like with the driveraccording to the scene of the selected scenario are stored. In the voicedata of the conversation by the agent, voice data to pose a question inorder to collect the driver information according to the presentembodiment are also stored.

[0136] Each data of the voice data 303 is specified by the charactermotion instruction data of the scene data.

[0137] In the image data 306, images to represent the figure of theagent to be used in each scene specified by the scenario, for example, afront facing agent image, a bowing image, a right-hand raising image,and the like are stored.

[0138] The figures of the agent stored in the image data 306 may notnecessarily be a human-like figure (male or female), and may be a figureof an animal as it is such as, for example, a chick, dog, cat, flog, ormouse, or a figure of personifiedly designed (illustrated) animal, andfurthermore may be a robot-like figure or a figure of a specificcharacter. Further, the agent does not need to stay in a fixed age, andmay be a figure that is initially a child, growing over time, andchanging (changing into a figure of an adult, then into a figure of anold person), by the learning function of the agent. In the image data306, these images of various figures of the agent, which can be selectedthrough the input apparatus 22 and the like according to the driver'spreferences, are stored.

[0139] The driver information 307 is the information on the driver, andused to make the behaviors by the agent better suit the driver'sdemands, tastes, and preferences.

[0140]FIG. 4 conceptually shows the composition of the driverinformation 307.

[0141] As shown in FIG. 4, the driver information 307 stores the basicdriver data such as the driver's ID (identification data), name, age,gender, marital state (married or not married), having or not having achild, the number of children, and the age of the children, as well asdata on hobbies and preferences.

[0142] The hobbies and preferences data are composed of the major itemssuch as sport, dining, and travel, and the detailed items included inthe major item. For example, a major item stores detailed data on thefavorite soccer team, favorite baseball team, interest in golf, and thelike.

[0143] In the present embodiment, priorities are set for each of thedriver information, and the agent poses a question to a driver on thedriver information not stored, in the order of descending priorities.The priority of the basic driver data is higher than that of the hobbiesand preferences data.

[0144] The questions to obtain the driver information are asked byoutputting the voice data specified in the character motion instructiondata in the question scenario stored in the scenario data file 302.

[0145] As the question sentences for the agent to obtain the driverinformation in the question scenario, the following sentences and thelike are prepared:

[0146] “What is your name?”

[0147] “Will you tell me your age?”

[0148] “Will you tell me your gender?”

[0149] “What is your hobby?”

[0150] “Are you interested in baseball?”

[0151] “Are you interested in golf?”

[0152] “Are you interested in soccer?”

[0153] “Are you interested in music?”

[0154] “Which genre of music do you like?”

[0155] “What is your favorite soccer team?”

[0156] “What is your favorite baseball team?”

[0157] “Do you like hamburger?”

[0158] “When you dine out, do you go to a family restaurant most of thetime?”

[0159] These questions are stored in the character motion instructiondata in the scenario data file 302 as the question sentences for eachquestion scenario.

[0160] The data of each question sentence is delimited by each voicedata stored in the voice data 302 in such a manner as, for example,“What is/your/favorite/baseball team?”. The question is asked by theagent to the driver by reading out a voice data corresponding to adelimited unit in the question sentence, and outputting it from thevoice output device 25.

[0161] The navigation data 31 stores, as the various data to be used forthe route guidance and so forth, various data such as a communicationregion data file, picturized map data file, intersection data file, nodedata file, road data file, search data file, photo data file, and thelike.

[0162] The communication region data file stores, by each type of cellphones, communication region data to be used to display on the displayapparatus 27 a region in which a cell phone used in the vehicle byconnecting or not connecting to the communication control 27 cancommunicate from the inside of the vehicle; or to use such acommunicatable region when searching the route.

[0163] The picturized map data file stores data of picturized maps whichare picturized in the display apparatus 27. This picturized map datastores hierarchized maps, for example, map data for each tier such as,beginning from the highest tier, Japan, Kanto District, Tokyo, andKanda. To each tier of the map data, a map code is respectively given.

[0164] In the intersection data file, the intersection number to specifyeach intersection, intersection name, coordinate of an intersection(longitude and latitude), number for a road whose starting point orendpoint is at the intersection, presence of stoplights, and so forthare stored as the intersection data.

[0165] In the node data file, node data composed of information such asthe longitude and the latitude to define a coordinate of each point ofeach road is stored. In other words, this node data is a data on onepoint along a road, and where a connector between nodes is defined as anarc, a road is represented by connecting between the plural node columnsby the arc.

[0166] The road data file stores the road number to specify each road,number for an intersection being a starting point or an end point,numbers for roads having the same starting point or end point, width ofa road, prohibition information such as prohibition of entry, photonumber for photo data which will be referred later, and the like.

[0167] The road network data composed of the intersection data, nodedata, and road data stored respectively in the intersection data file,node data file, and road data file are used to search a route.

[0168] In the search data file, intersection column data, node columndata and the like composing the routes generated by the route search arestored. The intersection column data includes information such as theintersection name, intersection number, number for a photo of acharacteristic view of the intersection, turning point, distance, andthe like. Further, the node column data includes information to indicatethe position of the node such as the east longitude and the northlatitude.

[0169] In the photo data file stored are photographs of a characteristicview or the like which is seen at each intersection or when travelingstraight, by relating them to the photo number thereof, in the form ofdigital, analogue, or negative film.

[0170] The operations of thus structured driver information collectingapparatus will be explained next.

[0171]FIG. 5 is a flowchart showing operations of the scenario automaticstartup determination process. The routine of the scenario automaticstartup determination process performs the automatic startupdetermination for all the scenarios registered in a predetermined periodof time (by the minute, for example). Further, when a change of a stateis deemed to be substantial (for example, when a destination is set, andso forth), the state determining unit 111 issues an event, and thescenario execution request determination unit 112 performs determinationon an automatic startup for all the scenarios when receiving the event.

[0172] The scenario execution request determining unit 112 in the agentprocessing unit 11 obtains the current state or a change of the statefrom various data provided by the state determining unit 111 (step 11),and performs determination on the scenario startup conditions (includingwhether the state of the load on the driver is in a low state or notwhich is the startup condition in the present embodiment) stored in theautomatic startup condition table of the scenario data file 302.Further, in the present embodiment, when the event is for executing aquestion scenario, the scenario drive unit 113 checks out the driverinformation which has not been collected in the driver information 307and the priorities thereof, and determines a question which correspondsto the driver information of the highest priority (step 12).

[0173] Subsequently, whether the automatic startup conditions arefulfilled is determined (step 13), and when the conditions are fulfilled(; Y), the scenario execution request determining unit 112 issues to thescenario drive unit 113 an execution request event for the scenariowhich fulfills the automatic startup conditions (step 14). On the otherhand, when the automatic startup condition is not fulfilled (step 13;N), a transition to the step 15 is made without issuing the scenarioexecution request event.

[0174] Subsequently, it is determined whether the determination on theautomatic startup conditions for all the scenarios is completed, and ifnot (step 15; N), a return to the step 11 is performed and thedetermination on the next scenario is carried out. When thedetermination on the all scenarios is completed (step 15; Y), thescenario execution request determination unit 112 completes theprocessing.

[0175]FIG. 6 is a flowchart showing operations of the scenario driveprocessing.

[0176] On receiving the scenario execution request event, the scenariodrive unit 113 executes the scenario drive processing, and starts up thescenario which is requested (step 21)

[0177] Subsequently, the scenario drive unit 113 determines, for eachscene of the started scenario, whether or not it is instructed to storethe selected result (step 22). If it is instructed to store the selectedresult (step 22; Y), such a result selected by the driver (a reply tothe question) is stored (step 23). Hence, in the case of a questionscenario, new driver information can be collected from the driver'sreply and stored in the driver information 307.

[0178] After storing the driver information (step 23), or when a storageinstruction is not given in the step 22 (; N), the processing isterminated.

[0179]FIG. 7 shows a transition of scene screens of a question scenarioon hobbies and preferences (on food) among the question scenarios, bypresenting each scene.

[0180] The scene screen includes, as shown in FIG. 7(A), an agentdisplay screen 51 to display an image of the agent (still image/movingimage), a word balloon screen 52 to display a text corresponding to thevoice of the agent, title screen 53, and scene display screen 54 todisplay image data specific to each scene (images of the actual imagedata, answer selecting button, and the like).

[0181] As shown in FIG. 7, when the question scenario on preferences andhobbies (on food) are started up, the agent processing unit 11 readsout, from the scenario data file 302, the screen layout data of thescene which is specified by the scene header at the beginning, displaysit on the display apparatus 27, and at the same time outputs from thevoice output apparatus 25 the question-asking voice corresponding to thequestion sentence.

[0182] Subsequently, the agent processing unit 11 specifies to the voicerecognition unit 142 a voice dictionary if the reply by the driver tothe question is inputted by voice. In the case of the first scene 0x0001shown in FIG. 7(a), the dictionary for voice recognition for fouranswering buttons 54 a, which are “Japanese food”, “Western-style food”,“Chinese food”, and “no particular preference”, is specified as thedictionary for the voice recognition.

[0183] In the case of the question scenario in FIG. 7, the agentprocessing unit 11 displays, in the scene 0x0001 shown in (a), “whatgenre of food do you like?” in the word balloon screen 52, and at thesame time outputs the voice corresponding to the presentation in theword balloon screen 52 from the voice output apparatus.

[0184] This scene of asking a question to the driver is followed by theplural scenes branched according to the answer by the driver (one ofFIG. 7(b) to (e)).

[0185] Accordingly, when the driver selects “Japanese food” in the sceneof (a), the agent processing unit 11 displays the scene screen of thebranched scene 0x0002 of FIG. 7(b). In this scene screen, “Japanesefood” which has been selected is displayed in the title screen 53, atthe same time the word balloon screen “You like Japanese food.” isdisplayed, and the corresponding voice is outputted. Additionally, inthe branched scene screen of FIG. 7(b), the actual image 54 b of theJapanese food is read out of the scenario data and displayed in thescene display screen 54.

[0186] Further, the agent apparatus unit 11 stores the driver's answer,for example, “Japanese food” in the hobbies and preferences data of thedriver information 307 as the driver information (FIG. 6, step 23).

[0187] Hence, the agent processing unit displays and outputs each sceneimage and voice specified by the scenario sequentially until the lastscene.

[0188] In the case of the question scenario of FIG. 7, the agentprocessing unit 111 terminates the action by the character through theagent because the scenes (b) to (e) are the last scene.

[0189] Further, the agent processing unit 11 terminates the processingwhen an end is selected by the driver through turning-off of theignition or selection of an ending button (step 15; Y), and when not tobe ended (step 15; N), returns to the step 11 and determines on anexecution of the subsequent scenario.

[0190] As has been explained, with the present embodiment, the questionscenario by the agent is performed, and the driver information isobtained from the answer thereto, so that the driver's operations ofinputting the data on himself/herself are reduced.

[0191] Additionally, since the question scenario by the agent stipulatesthe startup condition of the scenario to be the case of the load on thedriver being low, there is such effect as that the scenario does notplace a burden on the driving operations by the driver, and the driveris less vexed by the questions.

[0192] While one embodiment of the driver information collectingapparatus in the present embodiment has been explained so far, thepresent invention is not limited to the embodiment explained, andvarious modification thereto are possible within the range described ineach of the claims.

[0193] For example, in the embodiment which has been explained, isexplained the case to determine the state of the load on the driverbased on both the scenario startup conditions regarding the load on thedriving operations and the scenario startup conditions regarding thevexation felt by the driver, but the state of the load may be determinedbased on one of these cases.

[0194] Further, in the embodiment explained above, as the scenarioautomatic startup conditions to determine the state of the load on thedriver, (A) to (J) and (a) to (e) are cited and explained, but thepresent invention is not limited to this, and other conditions may beapplied as long as they are conditions related with the determination onthe load on the driver.

[0195] For example, conditions such as: the steering angle is above thepredetermined value; the wiper is moving at a higher-than-predeterminedspeed; the ringtone of the cell phone is detected; the cell phone is inuse; the vehicle speed V is (for example, 150) km per hour or faster;and so forth may be adopted as the case that the load is high.

[0196] Further, where a radio or television is installed on the vehicle,an automatic startup prohibition condition not to start up the questionscenario when the radio or the like is ON may be established, while, asa condition which is given priority thereto, the reception of acommercial may be set as a startup condition of the question scenario.In this case, a commercial detecting sensor to detect reception of acommercial should be provided.

[0197] Each of the conditions referred to above may be used by itself orin any combination with other conditions, as a condition for a specificscenario or a condition common for plural scenarios.

[0198] Further, in the embodiment explained above, the conditions tostart up each scenario regarding all the actions by the agent aredetermined by the state of vehicle, condition of the driver (includingthe state of the load), and the learning condition of the agent.

[0199] Further to this, the psychological distance between the agent andthe driver may be determined and the scenario startup conditionsaccording to the psychological distance may be added. As a result, thestartup conditions on a specific scenario are modified according to thephysiological distance between the agent and the driver.

[0200] Here, the physiological distance between the agent and the driveris expressed by such parameters as the degree of friendliness, degree ofself-confidence, degree of amenability, degree of morality, and soforth, and transforms itself by the accumulated past replies and thelike by the driver. For example, when words of appreciation are spokenfor the agent's action, the psychological state of the agent becomessuch that the friendliness is increased, and the suggestion such aswhether to play a CD becomes more frequent, but in contrast, when theagent is cursed, the psychological state of the agent becomes such thatthe friendliness is reduced, and the actions such as actively suggestingto open the window, play a CD, and so forth are reduced.

[0201] Further, it is also possible to make a change into a friendly andcasual way of speaking or a businesslike way of speaking, according tothe parameter value of the psychological distance. In this case, thecharacter motion instruction data (see FIG. 3) for each scene accordingto the parameter value for each psychological distance should be stored.

[0202] When the scenario startup conditions according to thepsychological distance between the agent and the driver are added asreferred to above, an agent psychology unit is added to the compositionof the agent processing unit shown in FIG. 2. For example, this agentpsychology unit determines the psychological state of the agent byreceiving an event on the result of the determination on the state fromthe state determining unit 111. Further, in the agent psychology unit,there are a process to change the parameter of the psychological stateof the agent, as well as a process to notify of the agent'spsychological state. Subsequently, the scenario drive unit 113 refers tothe psychological state of the agent, and drives the scenario (controlsthe transition of the scene).

INDUSTRIAL AVAILABILITY

[0203] With the present invention, the driver information can beautomatically collected through the communications between the agent andthe driver. Accordingly, the driver information inputting operations bythe driver are reduced.

1. A driver information collecting apparatus, comprising: a load-on-driver detector to detect a load on a driver; a question outputter to output a question on the driver when a low-load state is detected by said load-on-driver detector; and an information obtainer to obtain driver information based on a reply by the driver to the question by said question outputter.
 2. The driver information collecting apparatus according to claim 1, further comprising: a driver information storage to store driver information; a question information storage to store a question for obtaining said driver information, wherein said question outputter reads out, from said question information storage, a question which corresponds to the driver information not stored in said driver information storage, and outputs the question.
 3. The driver information collecting apparatus according to claim 1 or claim 2, wherein: as a state of a driver detected by said state detector, at least one of a state of driving operation and a state of equipment manipulation is detected; and the load on the driver determined by said load-on-driver determiner is at least one of a load on the driving operations and an extent of vexation felt by the driver.
 4. The driver information collecting apparatus according to claim 1, claim 2, or claim 3, further comprising an agent presenter to present an artificial pseudo-living organism in the vehicle, wherein said question outputter poses a question through at least one of a display screen and voice output. 